Burke County
Mean-Field Assisted Deep Boltzmann Learning with Probabilistic Computers
Chowdhury, Shuvro, Niazi, Shaila, Camsari, Kerem Y.
Despite their appeal as physics-inspired, energy-based and generative nature, general Boltzmann Machines (BM) are considered intractable to train. This belief led to simplified models of BMs with restricted intralayer connections or layer-by-layer training of deep BMs. Recent developments in domain-specific hardware -- specifically probabilistic computers (p-computer) with probabilistic bits (p-bit) -- may change established wisdom on the tractability of deep BMs. In this paper, we show that deep and unrestricted BMs can be trained using p-computers generating hundreds of billions of Markov Chain Monte Carlo (MCMC) samples per second, on sparse networks developed originally for use in D-Wave's annealers. To maximize the efficiency of learning the p-computer, we introduce two families of Mean-Field Theory assisted learning algorithms, or xMFTs (x = Naive and Hierarchical). The xMFTs are used to estimate the averages and correlations during the positive phase of the contrastive divergence (CD) algorithm and our custom-designed p-computer is used to estimate the averages and correlations in the negative phase. A custom Field-Programmable-Gate Array (FPGA) emulation of the p-computer architecture takes up to 45 billion flips per second, allowing the implementation of CD-$n$ where $n$ can be of the order of millions, unlike RBMs where $n$ is typically 1 or 2. Experiments on the full MNIST dataset with the combined algorithm show that the positive phase can be efficiently computed by xMFTs without much degradation when the negative phase is computed by the p-computer. Our algorithm can be used in other scalable Ising machines and its variants can be used to train BMs, previously thought to be intractable.
LLMs Accelerate Annotation for Medical Information Extraction
Goel, Akshay, Gueta, Almog, Gilon, Omry, Liu, Chang, Erell, Sofia, Nguyen, Lan Huong, Hao, Xiaohong, Jaber, Bolous, Reddy, Shashir, Kartha, Rupesh, Steiner, Jean, Laish, Itay, Feder, Amir
The unstructured nature of clinical notes within electronic health records often conceals vital patient-related information, making it challenging to access or interpret. To uncover this hidden information, specialized Natural Language Processing (NLP) models are required. However, training these models necessitates large amounts of labeled data, a process that is both time-consuming and costly when relying solely on human experts for annotation. In this paper, we propose an approach that combines Large Language Models (LLMs) with human expertise to create an efficient method for generating ground truth labels for medical text annotation. By utilizing LLMs in conjunction with human annotators, we significantly reduce the human annotation burden, enabling the rapid creation of labeled datasets. We rigorously evaluate our method on a medical information extraction task, demonstrating that our approach not only substantially cuts down on human intervention but also maintains high accuracy. The results highlight the potential of using LLMs to improve the utilization of unstructured clinical data, allowing for the swift deployment of tailored NLP solutions in healthcare.
OpenAI expands ChatGPT 'custom instructions' to free users
OpenAI's custom instructions feature that rolled out to ChatGPT Plus subscribers in July, is now available to all users. The addition of custom instructions puts a new setting in your ChatGPT profile on desktop or iOS, and applies that setting across all of your conversations. Instead of having to give ChatGPT certain instructions at the beginning of every new conversation, it automatically calibrates its responses based on descriptions you add in settings. Just fill out the fields in ChatGPT settings, and see how it adapts. The custom instructions section of settings contains two text entry fields.
How does ChatGPT work?
Google, Wolfram Alpha, and ChatGPT all interact with users via a single line text entry field and provide text results. Google returns search results, a list of web pages and articles that will (hopefully) provide information related to the search queries. Wolfram Alpha generally provides mathematically and data analysis-related answers. ChatGPT, by contrast, provides a response based on the context and intent behind a user's question. You can't, for example, ask Google to write a story or ask Wolfram Alpha to write a code module, but ChatGPT can do these sorts of things. Fundamentally, Google's power is the ability to do enormous database lookups and provide a series of matches.
The Documentary Film's Artificial Fan
The dots representing locations on this white map on this documentary I watched for a few seconds inspired me this documentary Artificial Fan (DAF) that I am writing about after writing about this yellow sticker that is about JS dependencies switching and swapping. Maybe these two topics are related because all I will be writing about in this story is to some extent also about computational journalism because it's about producing web content and fan engagement experiences and platforms as well as automated conversations in social media and all of these autonomously. The identification of the materials used in a documentary, materials like archives, interviewed people, showcased facts like documents, and the gathering of more information about these materials for the purpose of creating even more content about them and crediting the people and organizations behind these materials is the purpose of this documentary Fan bot that I am about to describe and implement. Then I could say that this documentary fan bot is about producing engaging content autonomously for the fanbase of the underlying film documentary that it covers for the goals of having more reliable sources and materials about this film documentary first streamed on video streaming platforms like Netflix, Discovery, โฆ. This Documentary Fan bot that I am trying to describe in this story is about how one could use a computer to take any produced documentary Like โฆ and produces autonomously rich web content and social media discourses from this film documentary with what we call artificial intelligence and its special custom field that is machine learning.
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Chen, Wenhu, Chen, Jianshu, Qin, Pengda, Yan, Xifeng, Wang, William Yang
Semantically controlled neural response generation on limited-domain has achieved great performance. However, moving towards multi-domain large-scale scenarios are shown to be difficult because the possible combinations of semantic inputs grow exponentially with the number of domains. To alleviate such scalability issue, we exploit the structure of dialog acts to build a multi-layer hierarchical graph, where each act is represented as a root-to-leaf route on the graph. Then, we incorporate such graph structure prior as an inductive bias to build a hierarchical disentangled self-attention network, where we disentangle attention heads to model designated nodes on the dialog act graph. By activating different (disentangled) heads at each layer, combinatorially many dialog act semantics can be modeled to control the neural response generation. On the large-scale Multi-Domain-WOZ dataset, our model can yield a significant improvement over the baselines on various automatic and human evaluation metrics.
Analysis of Railway Accidents' Narratives Using Deep Learning
Heidarysafa, Mojtaba, Kowsari, Kamran, Barnes, Laura E., Brown, Donald E.
Automatic understanding of domain specific texts in order to extract useful relationships for later use is a non-trivial task. One such relationship would be between railroad accidents' causes and their correspondent descriptions in reports. From 2001 to 2016 rail accidents in the U.S. cost more than $4.6B. Railroads involved in accidents are required to submit an accident report to the Federal Railroad Administration (FRA). These reports contain a variety of fixed field entries including primary cause of the accidents (a coded variable with 389 values) as well as a narrative field which is a short text description of the accident. Although these narratives provide more information than a fixed field entry, the terminologies used in these reports are not easy to understand by a non-expert reader. Therefore, providing an assisting method to fill in the primary cause from such domain specific texts(narratives) would help to label the accidents with more accuracy. Another important question for transportation safety is whether the reported accident cause is consistent with narrative description. To address these questions, we applied deep learning methods together with powerful word embeddings such as Word2Vec and GloVe to classify accident cause values for the primary cause field using the text in the narratives. The results show that such approaches can both accurately classify accident causes based on report narratives and find important inconsistencies in accident reporting.